Compact Representations of Markov Decision Processes and Their Application to Printer Management
نویسندگان
چکیده
Many planning problems can be framed as Markov decision processes (MDPs). In this paper we discuss situations where regularities in states and variables lead to compact MDPs, particularly when variables have many categories and strong interrelation. We develop techniques that generate optimal policies by exploiting regularities in MDPs. We illustrate these ideas with a real problem on management of printing clusters.
منابع مشابه
Accelerated decomposition techniques for large discounted Markov decision processes
Many hierarchical techniques to solve large Markov decision processes (MDPs) are based on the partition of the state space into strongly connected components (SCCs) that can be classified into some levels. In each level, smaller problems named restricted MDPs are solved, and then these partial solutions are combined to obtain the global solution. In this paper, we first propose a novel algorith...
متن کاملL2 Teachers’ Representations of Classroom Management Events: Variations across Experience Levels
Knowledge representation, defined as the way individuals structure their knowledge and cognitive processing of events and the associated sense-making processes, is believed to influence teachers’ reasoning/thinking skills. While extensively researched in mainstream teacher education, this line of inquiry is essentially lacking in the L2 teacher education literature. To fill some of the void, th...
متن کاملA Bayesian model decision support system: dryland salinity management application
Addressing environmental management problems at catchment scales requires an integrated modelling approach, in which key bio-physical and socio-economic drivers, processes and impacts are all considered. Development of Decision Support Systems (DSSs) for environmental management is rapidly progressing. This paper describes the integration of physical, ecological, and socio-economic components i...
متن کاملProbabilistic Knowledge-Based Programs
We introduce Probabilistic Knowledge-Based Programs (PKBPs), a new, compact representation of policies for factored partially observable Markov decision processes. PKBPs use branching conditions such as if the probability of φ is larger than p, and many more. While similar in spirit to valuebased policies, PKBPs leverage the factored representation for more compactness. They also cope with more...
متن کاملApplication of Markov Processes to the Machine Delays Analysis
Production and non-productive equipment and personnel delays are a critical element of any production system. The frequency and length of delays impact heavily on the production and economic efficiency of these systems. Machining processes in wood industry are particularly vulnerable to productive and non-productive delays. Whereas, traditional manufacturing industries usually operate on homoge...
متن کامل